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4.  CONTRACT  OR  GRANT  NUMBER:  DAAG29-83-K-0028 
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1  Work  Completed 


During  the  last  three  years  we  feel  we  have  been  fortunate  to  make  sev¬ 
eral  significant  contributions,  some  of  them  in  conjunction  with  long  and 
short  term  visitors  supported  by  this  contract.  We  have  been  able  to  unify 
many  past  results  in  this  area.  Due  to  their  fundamental  nature  and  their 
wide  applicability,  most  of  our  results  reach  across  several  topics.  Drs. 
Lev-Ari,  Bruckstein,  Wax  and  Rao  have  completed  their  theses  during  the 
contracting  period  and  have  written  a  large  number  of  publications. 

The  best  overview  of  our  research  is  provided  by  the  list  of  publications 
describing  the  research  supported  at  least  in  part  by  ARO. 

In  reference  to  the  list  of  publications,  the  prefix  P  denotes  published 
journal  papers,  C  denotes  published  conference  papers,  A  denotes  accepted 
papers,  and  R  denotes  papers  under  review. 

We  shall  briefly  organize  and  review  them  here  under  four  major  head¬ 
ings: 

'  ■“■•••*«  '  *  •••  />-.  '  . 

1>  Inverse  Scattering  Theory' 

2.  VLSI  and  Parallel  Architectures  in  Signal  Processing 

3.  Adaptive  Identification  and  Signal  Processing 


4.  Stochastic  System  Modeling  and  Realization. 


1.1  Inverse  Scattering  Theory 


The  recent  period  has  seen  a  culmination  of  our  work  in  inverse  scattering 
theory.  We  have  arrived  at  a  satisfactory  and  simple  understanding  of  the 
inverse  scattering  problem.  A  key  to  this  has  been  a  formulation  in  terms 
of  transmission  lines,  which  has  shown  that  an  algorithm  due  to  I.Schur 
(1917)  for  testing  analytic  functions  for  boundedness  is  the  most  natural 
way  to  solve  one-dimensional  inverse  problems.  We  have  shown  that  various 
differential  methods  for  inverse  scattering  are  related  to  nonlinear  evolu¬ 
tion  equations  of  the  Riccati-type  and,  in  the  discrete  case,to  compositions 
of  linear  fractional  transformations.  Furthermore,  it  was  found  that  the 
inverse  scattering  methodology  could  be  applied  to  several  other  problems 
e.g.  partial  realizations.  Finally  the  effect  of  noisy  scattering  data  on  the 
quality  of  the  recovery  of  medium  parameters  and  approximate  methods  of 
inversion  using  separation  of  overlapping  echoes  have  been  also  extensively 
studied.  The  results  are  reported  in  papers  P-9,  P-15,  A-7,  A-9,  A-10,  R-9 
C-6,  C-7,  C-10,  C-12,  C-13,  C-19. 

A  thesis  by  A.M.  Bruckstein  made  several  major  contributions  to  this 
area  and  abstract  (of  this  thesis  and  of  others  cited  below)  is  appended  to 
this  report. 
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1.2  VLSI  and  Parallel  Architectures  in  Signal  Pro¬ 


cessing 

The  Ph.D  theses  of  Sailesh  Rao  and  Todd  Citron  (to  be  submitted  shortly) 
made  major  contributions  to  this  area. 

We  have  obtained  exciting  results  on  a  new  class  of  numerically  stable, 
cascade,  orthogonal  digital  filters  that  are  well  suited  for  VLSI  implemen¬ 
tation. 

We  have  identified  a  class  of  algorithms  called  Regular  Iterative  Algo¬ 
rithms  which  can  be  systematically  implemented  on  processor  arrays  with 
local  interconnections.  We  have  also  shown  that  algorithms  implementable 
on  systolic  arrays  form  a  sub-class  of  these  algorithms. 

Another  application  has  been  to  a  new  algorithm  for  decoding  BCH 
error  correcting  codes,  and  more  generally  for  the  evaluation  of  the  gcd  of 
two  polynomials.  The  new  algorithm  is  much  better  suited  to  parallel  VLSI 
implementations  than  currently  known  methods. 

Publications  in  this  area  were  P-4,  P-11,  P-13,  C-10,  C-ll,  C-18,  C-23 

1.3  Adaptive  Identification  and  Signal  Processing 

Our  development  of  new  estimation  algorithm  also  led  to  several  results  in 
system  identification  and  signal  processing.  Applications  to  Communica- 
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tion  Systems  have  been  extensively  studied.  We  have  also  obtained  several 
results  on  Recursive-Least-Squares  identification  and  estimation.  Adaptive 
Filtering  algorithms  with  automatic  gain  control  were  also  obtained. 

This  activity  is  illustrated  by  papers  P-6,  P-7,  P-12,  P-17,  P-18,  R-10, 
C-8,  C-9,  C-14,  C-28,  C-31 

A  thesis  by  J.M.  Cioffi  was  completed. 

1.4  Stochastic  System  Modeling  and  Realization 

Proper  modeling  of  a  stochastic  system  can  lead  to  better  understanding 
of  the  phenomenon  and  properly  selected  models  can  simplify  associated 
estimation(filtering,  prediction  and  smoothing)  problems.  Several  results 
on  factorization  of  structured  matrices  were  also  obtained. 

This  work  was  largely  carried  out  by  Dr.  H.  Lev-Ari. 

Publications  in  this  area  were  P-1,  P-2,  P-3,  P-15,  P-19,  A-2,  A-4,  A-5^ 
R-4,  R-5,  R-6,  C-13,  C-29 
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ARO 

Published  Journal  Papers 

P-1  I.  Gohberg,  T.  Kailath  and  I.  Koltracht,  “Linear  Complexity  Algorithms  for  Semiseparable 
Matrices,”  Journal  Integral  Equations  and  Operator  Theory,  Vol.  8,  pp.  779-804, December  1985. 

P-2  T.  Kailath  and  H.  Lev-Ari,  “On  Mappings  Between  Covariance  Matrices  and  Physical  Systems,” 
in  Contemporary  Mathematics ,  ed.  B.  Datta,  Amer.  Math.  Socy.,  Vol.  47,  pp.  241-252,  Providence,  R.I., 
1985. 


P-3  T.  Kailath  and  L.  Ljung,  “Explicit  Strict  Sense  State-Space  Realizations  of  Nonstationary  Gaus¬ 
sian  Processes,"  Inter' l.  J.  Control.  Vol.  42,  no.  5,  pp.  971-988,  November  1985. 

P-4  S.  K.  Rao  and  T.  Kailath,  "VLSI  Arrays  for  Digital  Signal  Processing,  Pt.  I.  A  Model 
Identification  Approach  to  Digital  Filter  Realizations,"  IEEE  Trans.  Circuits  and  Systems,  pp.  1105- 
1117,  Vol.  CAS-32,  no.  11,  November  1985. 

P-5  T.  J.  Shan,  M.  Wax  and  T.  Kailath,  “On  Spatial  Smoothing  for  Direction-of- Arrival  Estimation  of 
Coherent  Sources,"  IEEE  Trans.  ASSP,  Vol.  ASSP-33,  no.  4,  pp.  806-811,  August  1985. 

P-6  J.  M.  Cioffi  and  T.  Kailath,  “An  Efficient  RLS,  Data-Driven  Echo  Canceller  for  Fast  Initialization 
of  Full-Duplex  Transmission,”  IEEE  Trans.  Communications,  Vol.  COM-33,  no.  7,  pp.  601-611,  July 
1985.  IBM  Research  Laboratory  Tech.  Rept.  RJ  4598,  San  Jose,  CA,  February  1985. 

P-7  J.  M.  Cioffi  and  T.  Kailath,  “Windowed  Fast  Transversal  Filters  Adaptive  Algorithms  with  Nor¬ 
malization,”  IEEE  Trans.  ASSP,  Vol.  ASSP-33,  no.  3,  pp.  607-625,  June  1985.  Also,  IBM  Research 
Laboratory  Tech.  Rept.  RJ4544,  San  Jose,  December  1984. 

P-8  T.  J.  Shan  and  T.  Kailath,  “Adaptive  Beamforming  for  Coherent  Signals  and  Interference."  IEEE 
ASSP,  Vol.  ASSP-33,  no.  3,  pp.  527-536,  June  1985. 
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P-9  A.  M.  Bruckstein,  B.  C.  L6vy  and  T.  Kailath,  “Differential  Methods  in  Inverse  Scattering,"  SIAM 
J.  Appl.  Math,  Vol.  45,  no.  2,  pp.  312-335,  April  1985. 

P-10  M.  Wax  and  T.  Kailath,  “Detection  of  Signals  by  Information  Theoretic  Criteria,"  IEEE  Trans, 
on  ASSP,  Vol.  ASSP-33,  no.  2,  pp.  387-392,  April  1985. 

P-11  T.  Kailath,  “Signal  Processing  in  the  VLSI  Era,"  in  Modern  Signal  Processing  and  VLSI ,  pp.  5- 
24,  ed.  by  S.  Y.  Kung,  H.  Whitehouse  and  T.  Kailath,  Prentice-Hall,  1985. 

P-12  J.  M.  Cioffi  and  T.  Kailath,  “An  Efficient  Exact-Least-Squares  Fractionally  Spaced  Equalizer 
Using  Intersymbol  Interpolation,”  IEEE  Trans.  Communications,  Vol  SAC-2,  no.  5,  pp.  743-756,  Sep¬ 
tember  1984. 

P-13  S.  R.  Rao  and  T.  Kailath,  “Orthogonal  Digital  Filters  for  VLSI  Implementations,"  IEEE  Trans. 
Circuits  &  Systems,  Vol.  CAS-31,  no.  11,  pp.  933-945,  November  1984. 

P-14  M.  Wax,  T-J.  Shan  and  T.  Kailath,  “Spatio-Temporal  Spectral  Analysis  by  Eigenstructure 
Methods,"  IEEE  Trans.  ASSP.  Vol.  ASSP-32,  no.  4,  pp.  817-827,  August  1984. 

P-15  P.  Dewilde  and  H.  Dym,  “Lossless  Inverse  Scattering,  Digital  Filters,  and  Estimation  Theory," 
IEEE  Trans,  on  Inform.  Thy.,  vol.  IT-30,  no.  4,  pp.  644-662,  July  1984. 

P-16  T.  Kailath  and  M.  Wax,  “A  Note  on  the  Complementary  Model  of  Weinert  and  Desai,"  IEEE 
Trans.  Autom.  Contr.,  Vol.  AC-29,  no.6,  pp.  551-552,  June  1984. 

P-17  H.  Lev-Ari,  J.  Cioffi  and  T.  Kailath,  “Least-Squares  Adaptive-Lattice  and  Transversal  Filters:  A 
Unified  Geometric  Theory,"  IEEE  Trans,  on  Inform.  Thy.,  (special  issue),  vol.  IT-30,  pp.  222-236, 
March  1984. 

P-18  J.  M.  Cioffi  and  T.  Kailath,  “Fast,  Recursive-Least-Squares  Transversal  Filters  for  Adaptive 
Filtering,"  IEEE  Trans.  ASSP,  vol.  ASSP-32,  no.  2,  pp,  304-337,  April  1984. 


P-19  H.  Lev-Ari  and  T.  Kailalth,  “Lattice  Filter  Parametrization  and  Modeling  of  Nonstationary 
Processes,"  IEEE  Trans.  Inform.  Thy.,  vol.  IT-30,  no.  1,  pp.  2-16,  January  1984. 


P-20  M.  Wax  and  T.  Kailath,  “Optimum  Localization  of  Multiple  Sources  in  Passive  Arrays,"  IEEE 
Trans.  ASSP,  Vol.  ASSP-31,  no.  5,  pp.  1210-1218,  October  1983. 

P-21  M.  Wax  and  T.  Kailath,  “Efficient  Inversion  of  Doubly  Block  Toeplitz  Matrix,"  IEEE  Trans. 
ASSP,  Vol.  ASSP-31,  no.  5,  pp.  1218-1221,  October  1983. 

P-22  Y.  Genin,  P.  Van  Dooren,  T.  Kailath,  J-M.  Delosme  and  M.  Morf,  "On  I- Lossless  Transfer  Func¬ 
tions  and  Related  Questions,"  J.  Lin.  Alg.  and  Applns.,  pp.  251-275,  vo.  50,  April  1983. 

P-23  W.  H.  Kwon,  A.  M.  Bruckstein  and  T.  Kailath,  "Stabilizing  State-Feedback  Design  via  the  Mov¬ 
ing  Horizon  Method,"  Int.  J.  Control,  vol.  37,  no.  3,  pp.  631-643,  March  1983. 
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Accepted  Papers 


A-l  T.  Kailath,  “A  Theorem  of  I.  Schur  and  Its  Impact  on  Modem  Signal  Processing,”  special  issue, 
Integral  Equations  and  Operator  Theory,  1986. 

A-2  I.  Gohberg,  T.  Kailath  and  I.  Koltracht,  “A  Note  on  Diagonal  Innovation  Matrices,”  IEEE  Trans. 
ASSP. 


A-3  H.  Lev-Ari  and  T.  Kailath,  ‘‘Triangular  Factorization  of  Structured  Hermitian  Matrices,”  Integral 
Equations  and  Operator  Theory,  special  issue  dedicated  to  I.  Schur,  December  1985. 

A-4  M.  Wax  and  T.  Kailath,  "Decentralized  Processing  in  Passive  Arrays,"  IEEE  Trans.  ASSP. 

A-5  A.  Bruckstein  and  T.  Kailath,  “Inverse  Scattering  for  Discrete  Transmission-Line  Models,"  SIAM 
Review. 

A-6  A.  M.  Bruckstein  and  T.  Kailath,  “Recursive  Limited  Memory  Filtering  and  Scattering  Theory," 
IEEE  Trans.  Inform.  Thy. 

A-7  A.  Bruckstein  and  T.  Kailath,  “An  Inverse  Scattering  Framework  for  Several  Problems  in  Signal 
Processing,”  ASSP  Magazine. 

A-8  T.  Kailath  and  I.  Koltracht,  “Matrices  with  Block  Toeplitz  Inverses,”  Linear  Algebra  and  Its 
Applications. 

A-9  A.  M.  Bruckstein,  T.  J.  Shan  and  T.  Kailath,  "The  Resolution  of  Overlapping  Echoes,"  IEEE 
Trans.  ASSP,  February  1984. 


A-10  A.  Bruckstein  and  T.  Kailath,  “Some  Matrix  Factorization  Identities  for  Discrete  Inverse  Scatter¬ 
ing,"  Linear  Algebra  and  Its  Applications. 


Papers  Under  Review 


R-l  R.  Roy,  A.  Paulraj  and  T.  Kailath,  “ESPRIT  --  A  Subspace  Rotation  Approach  to  Estimation  of 
Parameters  of  Cisoids  in  Noise,”  submitted  IEEE  Trans.  ASSP. 

R-2  R.  Roy,  A.  Paulraj  and  T.  Kailath,  “ESPRIT  -  A  Subspace  Rotation  Approach  to  Estimation  of 
Parameters  of  Cisoids  in  Noise,”  submitted  IEEE  ISCS,  San  Jose,  CA,  May  1986. 

R-3  A.  M.  Bruckstein,  T.  J.  Shan  and  T.  Kailath,  “A  Time-Domain  Signal  Resolution  Problem,”  sub¬ 
mitted  ICASSP  86,  Tokyo,  Japan. 

R-4  I.  Gohberg,  T.  Kailath  and  I.  Koltracht,  “Linear  Complexity  Parallel  Algorithm  for  Discrete-Time 
Wiener  Filters  with  Optimum  Lag,”  submitted  IEEE  Trans.  ASSP. 

R-5  I.  Gohberg,  T.  Kailath  and  I.  Koltracht,  “Efficient  Solution  of  Linear  Systems  of  Equations  with 
Recursive  Structure,”  submitted  Linear  Algebra  and  Its  Applications. 

R-6  I.  Gohberg,  T.  Kailath  and  I.  Koltracht,  “A  Note  on  Diagonal  Innovation  Matrices,”  submitted 
IEEE  Trans.  ASSP. 

R-7  M.  Wax  and  T.  Kailath,  “Novel  Eigenstructure  Methods  for  Parameter  Estimation  of  Superim¬ 
posed  Signals,”  submitted  IEEE  Trans.  ASSP. 

R-8  M.  Wax  and  T.  Kailath,  “Simultaneous  Detection  and  Estimation  of  Superimposed  Signals,”  sub¬ 
mitted  IEEE  Trans.  IT. 

R-9  A.  M.  Bruckstein,  I.  Koltracht  and  T.  Kailath,  “Inverse  Scattering  with  Noisy  Data,"  submitted 
SIAM  J.  on  Scientific  and  Statistical  Computing. 

R-10  G.  Sutton  and  J.  M.  Cioffi,  “An  Efficient  Least-Squares  Solution  to  the  Double-Talking  Problem 
in  the  Voice-Type  Echo  Canceller,"  submitted  IEEE  Trans.  Communications. 
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Published  Conference  Papers 


C-l  D.  Spielman,  A.  Paulraj  and  T.  Kailath,  “Performance  Analysis  of  the  MUSIC  Algorithm,” 
ICASSP,  Tokyo,  Japan,  April  1986. 

C-2  D.  Spielman,  A.  Paulraj  and  T.  Kailath,  “Eigenstructure  Approach  to  Directions-of- Arrival  Esti¬ 
mation  in  IR  Detector  Arrays,”  ICASSP ,  Tokyo,  Japan,  April  1986. 

C-3  T.  J.  Shan,  A.  Paulraj  and  T.  Kailath,  “On  Smoothed  Rank  Profile  Tests  in  Eigenstructure 
Approach  to  Directions-of-Arrival  Estimation,”  ICASSP,  Tokyo,  Japan,  April  1986. 

C-4  R.  Roy,  A.  Paulraj  and  T.  Kailath,  “Direction-of- Arrival  Estimation  by  Subspace  Rotation 
Met*'  -  ESPRIT,”  ICASSP,  Tokyo,  Japan,  1986. 

C-5  D.  Spielman,  A.  Paulraj  and  T.  Kailath,  “A  High  Resolution  Algorithm  for  Combined  Time-of- 
Arrival  and  Direction-of- Arrival  Estimation,”  19th  Asilomar  Conference  on  Circuits,  Systems  and  Com¬ 
puters,  pp.  ,  Monterey,  CA,  November  6,  1985. 

C-6  T.  J.  Shan,  A.  M.  Bruckstein  and  T.  Kailath,  “Multiple  Signal  Resolution  with  Uncertain  Signal 
Subspace  -  A  Self-Cohering  Approach,”  19th  Asilomar  Conference  on  Circuits,  Systems  and  Comput¬ 
ers,  pp.  ,  Monterey,  CA,  November  6,  1985. 

C-7  T.  J.  Shan,  A.  M.  Bruckstein  and  T.  Kailath,  “Adaptive  Resolution  of  Overlapping  Echoes,”  19th 
Asilomar  Conference  on  Circuits,  Systems  and  Computers,  pp. ,  Monterey,  CA,  November  6,  1985. 

C-8  T.  J.  Shan  and  T.  Kailath,  “Adaptive  Filtering  Algorithms  with  Automatic  Gain  Control,”  19th 
Asilomar  Conference  on  Circuits,  Systems  and  Computers,  pp.  ,  Monterey,  CA,  November  6,  1985. 

C-9  J.  M.  Cioffi  and  T.  Kailath,  “An  Efficient  RLS,  Data-Driven  Echo  Canceller  for  Fast  Initialization 
of  Full-Duplex  Transmission, “  ICC  1985,  Chicago,  ILL,  June  1985.  IBM  Research  Laboratory  Tech. 
Rept.  RJ  4598,  San  Jose,  CA,  February  1985. 


C-10  T.  K.  Citron,  A.  M.  Bruckstein  and  T.  Kailath,  “An  Inverse  Scattering  Approach  to  the  Partial 
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Wave  propagation  in  a  layered,  one-dimensional,  scattering  medium  is 
interpreted  as  a  model  for  •  various  signal  processing  algorithms.  The 
computational  methods  for  determining  the  propagated  waves  from  the  ones 
launched  into  the  medium  at  boundaries  and  the  changes  in  overall  medium 
properties  under  cascading  of  scattering  layers  with  given  properties  form  the 
conceptual  basis  shared  by  many  such  algorithms.  * . .  '  - ‘  ‘  " 

In  filtering  of  signals  with  given  state-space  models,  the  Hamiltonian 
formulation  suggests  that  the  estimates  and  the  adjoint  variables  may  be 
regarded  as  waves  propagating  in  a  medium  extending  in  time,  with  local 
properties  defined  by  the  time  varying  signal  model.  Using  this  interpretation  a 
new  algorithm  for  limited-memory  Kalman  filtering  is  derived. 

Physical  scattering  theory  deals  with  the  more  general  situation  of  infinite¬ 
dimensional  vectors  or  time  functions  propagating  in  a  medium  that  extends  in 
space.  In  this,  spatio-temporal  setting,  the  structure  of  parametrized  media  may¬ 
be  adjusted  to  model  a  variety  of  objects,  from  a  general  linear  system  to 
nonuniform  transmission-lines,  geophysical  layered-earth  structures  and  wave- 
digital  filters.  Inverse  problems,  requiring  the  determination  of  medium  properties 
from  its  response  to  a  probing  signal,  are  shown  to  be  solvable,  for  a  variety  of 

-  v  - 


medium  structures,  by  either  layer-peeling  or  layer-adjoining  algorithms.  These 
recursive  inverse  scattering  procedures,  derived  by  simple  causality  arguments, 


are  computationally  efficient  since,  in  some  sense,  they  let  the  medium  perform 
the  inversion  by  itself  and  thus  directly  and  fully  exploit  the  structural 
information  about  it.  When  applied  to  various  scattering  structures,  these 
inversion  algorithms  provide  alternative  methods  of  deconvolution,  nested  partial 
realization,  geophysical  prospecting,  and  synthesis  of  nonuniform  transmission- 
lines  and  digital  filters. 

Causality  and  symmetry'  principles  are  shown  to  provide  a  unified  derivation 
of  the  classical  methods  for  solving  inverse  problems.  These  methods  proceed  via 
solutions  of  nested  sets  of  integral  (matrix)  equations.  In  this  context  we  make 

■  •  ••  •  •  t  •  l  • '  ■  -  v  *<• . .  •  ' -  • 

the  connection  to  fast  estimation/factorization  algorithms,  by  showing  that  the 
recursive  procedures  derived  in  the  literature  exploit  matrix  structures  to 
indirectly  arrive  at  layer-adjoining  algorithms.  Some  practical  issues  concerning 
inverse  scattering  with  noisy  data,  and  a  new  algorithm  for  inversion  which 
combines  noise  propagation  results  with  prior  information  on  parameter  profiles, 
are  also  analyzed. 
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ABSTRACT 


The  dissertation  presents  a  general  theory  of  constant- parameter  modular  lattice 
models  for  nonstationary,  discrete-time,  second-order  processes.  This  theory  spans  a 
wide  range  of  topics,  beginning  with  the  derivation  of  a  canonical  parametrization  of 
nonstationary  covariances,  proceeding  through  covariance  extension  and  interpolation 
problems,  and  culminating  in  the  construction  of  lattice-form  modeling  and  whitening 
filters. 

The  canonical  parametrization  of  nonstationary  covariances  involves  Schur 
coefficients,  which  coincide  with  the  well-known  reflection  (or  PARCOR)  coefficients 
when  the  covariance  is  stationary,  and  the  newly  introduced  congruence  coefficients, 
which  are  necessary  to  complete  the  parametrization.  The  congruence  coefficients  also 
provide  the  time-varying  gains  of  a  tapped-delay-time  realization  of  the  whitening  filter 
for  the  process.  A  constant-parameter  realization  of  the  same  filter  can  be  derived  by 
combining  a  lattice  filter  structure  with  a  tapped-delay-line,  both  with  time-invariant 
gains.  This  configuration  also  provides  a  recursive  relation  for  the  congruence  coefficients 
(namely,  a  generalized  Levinson-Szego  recursion). 

The  tapped-delay-line  part  of  the  lattice-filter  model  can  be  eliminated  for 
covariances  that  possess  a  newly  defined  property  called  admissibility.  The 
parametrization  of  admissible  covariances  involves  the  Schur  coefficients  alone,  in 
analogy  to  stationary  covariances,  which  are  completely  characterized  by  their  PARCOR 
coefficients.  VVe  derive  an  explicit  expression  for  the  congruence  coefficients  of  admissible 
covariances  in  terms  of  the  Schur  coefficients.  Admissibility  also  plays  a  central  role  in 
the  derivation  of  Christoffel-Darboux-type  formulas  for  inversion  of  covariances,  and 
provides  a  convenient  classification  of  all  covariances  into  four  basic  types. 


The  construction  of  constant-parameter  lattice  models  is  made  possible  by 
developing  a  natural  connection  between  the  displacement  structure  property  of  a 
covariance  matrix  and  Schur’s  test  for  positive  semi-definiteness  of  matrices.  We  also 
extend  the  displacement  structure  property,  thereby  expanding  the  class  of 
nonstationary  covariances  that  possess  a  lattice-form  model. 

Several  properties  that  are  commonly  associated  with  stationary  covariances  are 
shown  to  be  exhibited  also  by  certain  nonstationary  processes.  In  addition  to  constant- 
parameter  lattice  models  and  Schur-coefficient  parametrization,  which  were  already 
mentioned  above,  these  include  the  minimum-phase  property  of  whitening  filters,  the 
maximum-entropy  property  of  finite-order  interpolants,  and  the  existence  of 
(asymptotic)  spectral  density  functions.  In  particular,  we  present  a  family  of 
nonstationary  covariances  that  possess  all  the  properties  mentioned  above. 

From  a  methodological  point  of  view,  the  dissertation  introduces  a  transform- 
domain  approach,  in  contrast  to  the  time-domain  approach  used  in  previous  work  on 
nonstationary  lattice-filter  modeling.  Many  of  the  new  results  in  this  dissertation  have 
been  stimulated  by  the  transform-domain  interpretation  of  known  results.  The  simplicity 
of  the  transform-domain  derivation  of  these  new  results  exhibits  the  utility  of  our 
approach. 
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ABSTRACT 

This  dissertation  is  about  Regular  Iterative  Algorithms,  their  properties,  their 
implementations  on  processor  arrays  and  their  applications.  It  is  shown  that  there 
exists  a  sub-class  of  these  Regular  Iterative  Algorithms  that  is  isomorphic  with 
systoliclwavefront  arrays.  Given  an  algorithm  that  belongs  to  this  sub-class,  certain 
procedures  are  devised  for  obtaining  a  variety  of  systolic  array  architectures  for 
implementing  the  algorithm.  Among  the  infinitely  many  such  arrays  that  can  be 
designed,  some  pointers  are  provided  for  making  a  rational  choice,  based  on  some 
objective  criteria. 

If  the  given  Regular  Iterative  Algorithm  is  not  necessarily  in  this  sub-class, 
then  some  procedures  are  presented  for  determining  asymptotic  lower-bounds  on 
the  time  required  for  executing  the  algorithm,  and  on  the  amount  of  storage  neces¬ 
sary  during  the  execution.  Further,  it  is  shown  how  one  can  derive  processor  array 
implementations  of  the  algorithm  which  meet  the  predicted  lower  bounds.  However, 
the  resulting  processor  arrays  are  not  systolic  as  per  the  definition  used  here,  but 
can  always  be  described  as  regular  interconnections  of  similar  processing  elements, 
together  with  register  pipelines,  all  operating  in  a  globally  synchronous  fashion. 

The  procedures  described  for  obtaining  these  lower-bounds  and  for  designing 
an  implementation  of  the  algorithm,  can  be  executed  in  constant  time  on  some 
sequential  back-end  processor,  independent  of  the  number  of  computations  in  the 
Regular  Iterative  Algorithm.  Furthermore,  the  processor  arrays  derived  for  imple¬ 
menting  the  algorithm  can  be  simulated  on  a  fixed  torus-connected  architecture,  thus 
opening  up  the  possibility  of  designing  a  compiler  that  takes  as  input  a  high-level 
description  of  the  Regular  Iterative  Algorithm  and  outputs  the  program  that  must  be 
repeatedly  executed  by  each  processor  in  such  an  architecture. 

The  usefulness  of  Regular  Iterative  Algorithms  for  various  problems  in  signal 
processing,  graph  theory,  numerical  linear  algebra  and  other  scientific  applications 
is  also  demonstrated  in  this  thesis.  In  most  cases,  these  Regular  Iterative  Algo¬ 
rithms  can  be  derived  by  reformulating  existing  ones  using  some  simple  heuristic 
rules.  For  some  problems,  such  as  digital  filtering  and  the  problem  of  determining 
the  transitive  closure  of  a  graph,  it  is  shown  to  be  advantageous  to  consider  the 
problem  afresh  and  synthesize  Regular  Iterative  Algorithms  for  them,  from  first 
principles. 
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ABSTRACT 

Adaptive  transversal  filters  have  been  proposed  as  solutions  for  many  real-time 
linear-estimation  problems  in  communications,  such  as  channel  equalization,  echo 
cancellation  and  antenna-array  beamforming.  Recursive-Least-Squares  (RLS)  adaptive 
algorithms,  which  have  been  implemented  in  lattice  as  well  as  transversal- filter  form, 
have  been  shown  to  exhibit  excellent  convergence  and  tracking  properties  in  these  same 
applications.  However,  these  RLS  algorithms  have  suffered  from  high  computational 
requirements  and/or  poor  numerical  (limited-precision)  performance  in  comparison  to  the 
widely  used,  but  slower-converging,  stochastic-gradient  or  Least-Mean-Square  (LMS) 
adaptive  algorithms.  In  this  dissertation,  transversal  filters  are  further  examined  in  the 
solution  of  the  RLS  problem  to  render  computational  cost  competitive  with  the  LMS 
algorithms  and  to  provide  acceptable  numerical  performance  for  these  real-time 
applications.  These  improvements  are  achieved  while  simultaneously  maintaining 
complete  compatibility  with  the  widely-used  stochastic-gradient  (LMS)  algorithms. 

We  present  new  Fast  Transversal  Filters  (FTF)  adaptive  algorithms.  The  FTF 
algorithms  reduce  computation  by  a  factor  of  at  least  two,  and  in  many  cases  by  more 
than  an  order  of  magnitude,  in  comparison  to  the  most  efficient,  existing,  RLS 
algorithms  (thus,  the  name  “Fast").  Further  comparison  reveals  computational 
requirements  of  the  FTF  algorithms  that  range,  depending  on  the  application,  from  .5  to 
3.5  times  those  of  the  slower-learning,  stochastic-gradient  algorithms.  Various  common 
windows  are  considered  for  the  RLS  criterion,  and  the  FTF  algorithms  are  extended  to 
the  so-called  “covariance”  windows,  as  well  as  to  a  more  general  window,  by  recursively 
propagating  additional  transversal  filters.  We  also  investigate  and  solve  numerical- 
instability  problems  that  arise  both  during  initialization  and  during  steady-state  (real- 
time)  operation  of  the  algorithms.  The  solutions  include  modifications  of  the  FTF 
algorithms  to  include  soft-constraints  on  the  performance  criteria,  rescue  operations,  and 
dynamic-range-increasing  normalized  (square-root)  FTF  algorithms.  Conceptually,  all 
improvements  are  attained  through  various  uses  of  a  single  geometric  formula  that  is  a 
generalizat  ion  of  the  Pythagorean  Theorem. 

Simulations  are  presented  to  verify  the  performance  and  improvements  of  the  FTF 
algorithms.  Specific  results  are  presented  for  the  applications  of  a  fractionally  spaced 
equalizer,  a  data-driven  echo  canceller,  and  a  voice-type  echo  canceller. 
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